pyarrow: Add version 16.1.0 - #3318

Arrow 16.1.0 pins Abseil 20211102.0 for its BUNDLED absl_ep. That
release's absl/strings/internal/str_format/extension.h forward-declares
`enum class FormatConversionChar : uint8_t` and
`enum class FormatConversionCharSet : uint64_t` without including
<cstdint>, relying on libstdc++ headers to pull it in transitively.
Since GCC 13, libstdc++ no longer does, and manylinux_2_39_riscv64 ships
GCC 14, so both declarations fail and str_format's extension.cc and
float_conversion.cc cascade into "FormatConversionCharSet was not
declared" / "no member named conversion_char". Abseil added the include
upstream in later LTS releases; 25.0.1/26.0.0 pin 20250127.0 and never
hit it.

Pass -DCMAKE_CXX_FLAGS="-include cstdint": Arrow copies CMAKE_CXX_FLAGS
into EP_CXX_FLAGS, which every bundled ExternalProject receives through
EP_COMMON_CMAKE_ARGS, so absl_ep (and gRPC/ORC built on top of it) get
the header without patching Arrow or Abseil. Same remedy as
build-aimrocks.yml uses for the same libstdc++ change.

pyarrow 16.1.0's setup.py does `import pkg_resources` at module scope
(and uses pkg_resources.resource_filename in build_ext). setuptools
82.0.0 removed pkg_resources, and the unpinned preinstall resolved to
84.2.1, so `build --no-isolation` died in get_requires_for_build_wheel
with ModuleNotFoundError. Cap at <82 (81.0.0 still ships it), as
asyncpg does (gotcha 29).

16.1.0's setup.py sets use_scm_version's write_to to
os.path.join(setup_dir, 'pyarrow/_generated_version.py') with an
abspath'd setup_dir. setuptools_scm 10.1 added _sanitize_relative_path
to its build_py hook and rejects that with "Version file path must be
relative"; 9.2.2 accepts an absolute write_to under root by design.

Cython 3.1 (May 2025) tightened with-nogil GIL-safety checking;
pyarrow/table.pxi's `options.unify_schemas = promote_options != "none"`
inside a nogil block is a Python string compare that only Cython's
older, more lenient check let through. Upstream's own
build-system.requires floor for 16.1.0 is cython>=0.29.31 -- release-era
Cython was 3.0.x. Scoped to 16.1.0 alone: the matrix already excludes
cp313/cp314/cp314t for it, and those are the only legs that need 3.1's
free-threading support.

16.1.0's bundled conftest.py defines pytest_ignore_collect(path, config),
the pre-9.1 hookspec signature; pytest 9.1 dropped the legacy `path`
LEGACY_PATH parameter, so pluggy's hookimpl validation rejects it at
collection time with "Argument(s) {'path'} are declared in the hookimpl
but can not be found in the hookspec". Confirmed by diffing pytest's
hookspec.py across 7.4.4/8.x/9.0.0/9.1.0: the signature holds the legacy
path parameter through 9.0.0 and drops it at 9.1.0. Same release-era-drift
shape as the earlier cython cap, this time in the test phase's floating
pytest install rather than the build tool; scoped to 16.1.0 alone. Needed
PYARROW_VERSION threaded into the test step's docker run, which previously
only had it available in the outer job env, not inside the container.

The <9.1 cap wasn't low enough. pytest escalates a RemovedInN warning
to a hard error starting at major version N: confirmed with a two-line
repro plugin exercising 16.1.0's legacy pytest_ignore_collect(path,
config) hook under both 8.4.0 and 9.0.3 -- 8.4.0 only warns
(PytestRemovedIn9Warning) and collection passes, 9.0.3 raises that
exact warning as an error during plugin registration. pyarrow's own
pytest config (python/setup.cfg) only errors on a SparseDataFrame
FutureWarning, so the escalation is pytest's own default, not
something pyarrow's filterwarnings adds. Confirmed again from the
actual CI log: pytest 9.0.3 (resolved under the old <9.1 cap) fails
collection with exactly this PytestRemovedIn9Warning-as-error.

….1.0

Two independent, confirmed-from-the-CI-log root causes for the 46 test
failures once collection started passing:

- pandas 3.0 made string-backed dtypes the default for columns that
  used to come back as plain object dtype, and changed
  pa.array(Categorical)'s inferred value type to large_string. 16.1.0's
  pandas-compat code and its own test suite's expected values both
  predate that, so the unconstrained `pip install pandas` resolving to
  today's 3.1.0 broke ~42 tests purely on dtype-equality assertions
  (test_backwards_compatible_index_naming: dtype='object' vs 'str';
  test_dictionary_from_pandas: string vs large_string). Capped to
  pandas<3, same release-era-drift shape as the earlier cython/pytest
  caps, this time in a floating test dependency.

- test_memory.py hardcodes `should_have_jemalloc = sys.platform ==
  "linux"` (removed in later pyarrow releases -- it's gone by 25.0.1)
  and never checks whether this build actually enabled jemalloc. We
  build with ARROW_JEMALLOC=OFF ("jemalloc is page-size fragile"), so
  4 tests fail on that wrong assumption regardless of pandas:
  ArrowNotImplementedError, "Unsupported backend 'jemalloc'",
  "'jemalloc' in ['mimalloc', 'system']", and an allocation byte-count
  tied to the allocator. Excluded by test name via `-k` (not
  --deselect: --pyargs resolves test files under the interpreter's own
  site-packages path, which varies per matrix.python).

Both scoped to 16.1.0 alone.